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DETAILED ACTION 

1. This is the first Office action regarding application number 10/543,079 which was filed 
on 01/20/2004. 

Claim Rejections - 35 USC § 101 

2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or 
composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, 
subject to the conditions and requirements of this title. 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or 
composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, 
subject to the conditions and requirements of this title. 

The USPTO "Interim Guidelines for Examination of Patent Applications for Patent 
Subject Matter Eligibility" (Official Gazette notice of 22 November 2005), Annex IV, reads as 
follows: 

Descriptive material can be characterized as either "functional descriptive material" or 
"nonfunctional descriptive material." In this context, "functional descriptive material" consists of 
data structures and computer programs which impart functionality when employed as a computer 
component. (The definition of "data structure" is "a physical or logical relationship among data 
elements, designed to support specific data manipulation functions." The New IEEE Standard 
Dictionary of Electrical and Electronics Terms 308 (5th ed. 1993).) "Nonfunctional descriptive 



Application/Control Number: 10/543,079 Page 3 

Art Unit: 2622 

material" includes but is not limited to music, literary works and a compilation or mere 
arrangement of data. 

When functional descriptive material is recorded on some computer-readable medium it 
becomes structurally and functionally interrelated to the medium and will be statutory in most 
cases since use of technology permits the function of the descriptive material to be realized. 
Compare In re Lowry, 32F.3dl579, 1583-84, 32 USPQ2d 1031, 1035 (Fed. Cir. 1994) (claim to 
data structure stored on a computer readable medium that increases computer efficiency held 
statutory) and Warmerdam, 33 F.3d at 1360-61, 31 USPQ2d at 1759 (claim to computer having a 
specific data structure stored in memory held statutory product-by-process claim) with 
Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 1760 (claim to a data structure per se held 
nonstatutory). 

In contrast, a claimed computer-readable medium encoded with a computer program is a 
computer element which defines structural and functional interrelationships between the 
computer program and the rest of the computer which permit the computer program's 
functionality to be realized, and is thus statutory ( See Lowry, 32 F.3d at 1583-84, 32 USPQ2d at 
1035). 



3. Claims 16-18 are rejected under 35 U.S.C. 101 because claims 16-18 recites "an image 
processing program for processing, by means of a computer" the program itself is not a process; 
therefore the invention as claimed is non-statutory. The claimed invention is directed to non- 
statutory subject matter. 
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Claim Rejections - 35 USC §102 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for 
patent or (2) a patent granted on an application for patent by another filed in the United 
States before the invention by the applicant for patent, except that an international 
application filed under the treaty defined in section 351(a) shall have the effects for 
purposes of this subsection of an application filed in the United States only if the 
international application designated the United States and was published under Article 
21(2) of such treaty in the English language. 

4. Claims 1, 12, and 15-16 are rejected under 35 U.S.C. 102(e) as being anticipated by 

Kakarala (US 7,015,961 B2). 

Regarding claim 1, Kakarala teaches an image pickup system for processing an image 
signal at each pixel which is composed of more than one color signals and one or more of the 
color signals are dropped out according to the location of the pixel (see Figs. 8 and 9, column 11, 
lines 1-67) comprising: first interpolation means for interpolating the color signals dropped-out 
from the image signals by a first interpolation method (see Figs. 9, the first interpolation takes 
place in element 600; column 11, lines 1-67); precision verification means for verifying the 
interpolation precision on the basis of the image signals and the color signals interpolated by the 
first interpolation means (see Figs. 9, the verification means 920 of Fig. 9 checks if there are any 
defective pixels i.e. if the first interpolation is sufficient or insufficient; column 11, lines 1-67); 
and second interpolation means for interpolating the color signals dropped-out from the image 
signals by a second interpolation method that is different from the first interpolation method in 
cases where it is judged that the interpolation precision by the first interpolation method is 
insufficient (see Figs. 9, once the verification means 920 checks the precision of the first 



Application/Control Number: 10/543,079 Page 5 

Art Unit: 2622 

interpolation and there are still defective pixels, it does a second interpolation in step 930 and 
the process continues; column 11, lines 1-67). 

Regarding claim 12, Kakarala further teaches control means that can control such that 
the operation of the precision verification means and the operation of the second interpolation 
means are stopped (see Figs. 9, once the verification means 920 checks the precision of the first 
interpolation and there are no defective pixels, it does not perform a second interpolation in step 
930 if there are no defective pixels in step 920; when it finally outputs the pixels without any 
defect that have gone through the first interpolation process in step 910; the verification means 
and the second interpolations means are not activated i.e. stopped). 

Regarding claim 15, Kakarala further teaches the control means comprises information 
acquisition means for acquiring at least one type of information selected from a set comprising 
image quality information relating to the image quality of the image signals means (see Figs. 9, 
once the verification means 920 checks the precision of the first interpolation and there are still 
defective pixels, by checking whether there are defective pixels or not, it is checking the quality 
of the image in step 920; column 11, lines 1-67. The output 940 is selected when it finally 
outputs the pixels without any defect that have gone through the first interpolation process in 
step 910), image pickup mode information set in the image pickup system, and interpolation 
processing switching information that can be manually set, and judgment means forjudging 
whether or not the operations are to be stopped on the basis of at least one type of information 
selected from a set comprising the image quality information, image pickup mode information, 
and interpolation processing switching information means (see Figs. 9, once the verification 
means 920 checks the precision of the first interpolation and there are still defective pixels, it 
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does not perform a second interpolation in step 930 if there are no defective pixels in step 920; 
column 11, lines 1-67. The output 940 is selected when it finally outputs the pixels without any 
defect that have gone through the first interpolation process in step 910). 

Regarding claim 16, all the limitations are addressed in claim 1. Moreover, Kakarala 
discloses an image processing program for processing an image signal at each pixel which is 
composed of more than one color signals and one or more of the color signals are dropped out 
according to the location of the pixel (see column 5, lines 10-25). 



Claim Rejections - 35 USC §103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

6. Claims 4-7 and 9 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

Kakarala (US 7,015,961 B2) in view of Maenaka (5,552,827). 

Regarding claim 4, Kakarala teaches the first interpolation means or second 
interpolation means comprises, interpolation signal calculation means for calculating a plurality 
of interpolation signals relating to predetermined directions from the pixels of interest within the 
regions, 
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Kakarala does not teach extraction means for extracting regions of a predetermined size 
centered on pixels of interest from the image signals, edge extraction means for extracting a 
plurality of edge intensities relating to predetermined directions from the pixels of interest within 
the regions, weighting calculation means for calculating weighting coefficients that are 
normalized from the edge intensities and calculation means for calculating the dropped-out color 
signals in the pixels of interest on the basis of a plurality of weighting coefficients relating to the 
predetermined directions and a plurality of interpolation signals relating to the predetermined 
directions 

However, Maenaka et al. teaches extraction means for extracting regions of a 
predetermined size centered on pixels of interest from the image signals, edge extraction means 
for extracting a plurality of edge intensities relating to predetermined directions from the pixels 
of interest within the regions (see column 1, lines 1-57; column 2, lines 1-35), weighting 
calculation means for calculating weighting coefficients that are normalized from the edge 
intensities and calculation means for calculating the dropped-out color signals in the pixels of 
interest on the basis of a plurality of weighting coefficients relating to the predetermined 
directions and a plurality of interpolation signals relating to the predetermined directions (see 
Fig. 2, element 74 and column 1, lines 1-57; column 2, lines 1-35). 

Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to modify Kakarala with the teachings of Maenaka et al. to extract a 
plurality of edge intensities relating to a predetermined direction from the regions of interest and 
for calculating the dropped-out color signals. The motivation to do so would have been to 
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prevent a false color signal from being produced as taught by Maenaka et al. (see column 2, lines 
1-7). 

Regarding claim 5, Kakarala as modified further teaches the first interpolation means or 
second interpolation means comprises extraction means for extracting regions of a predetermined 
size centered on pixels of interest from the image signals (see Kakarala Fig. 9 column 2, lines 1- 
35 and column 3, lines 45-55), and calculation means for calculating the dropped-out color 
signals in the pixels of interest within the regions by linear interpolation or cubic interpolation 
(see Kakarala column 1, lines 25-40). 

Regarding claim 6, Kakarala as modified further teaches characterized in that the first 
interpolation means or second interpolation means comprises extraction means for extracting 
regions of a predetermined size centered on pixels of interest from the image signals (see 
Kakarala Fig. 9 column 2, lines 1-35 and column 3, lines 45-55), correlation calculation means 
for determining as a linear equation the correlation between the respective color signals within 
the regions as a linear equation, and calculation means for calculating the dropped-out color 
signals on the basis of the image signals and the correlation (see Maenaka Fig.l, element 60; 
column 2, lines 1-54; and column 4, lines 1-15; column 8, lines 1-55). 

Regarding claim 7, Kakarala as modified further teaches the precision verification 
means comprises correlation calculation means for determining correlation information relating 
to the correlations between the respective color signals for each predetermined region on the 
basis of the image signals and the color signals interpolated by the first interpolation means (see 
Kakarala Figs. 9, the verification means 920 of Fig. 9 checks if there are any defective pixels i.e. 
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if the first interpolation is sufficient or insufficient; column 11, lines 1-67), and correlation 
verification means for verifying the interpolation precision on the basis of the correlation 
information (see Maenaka Fig.l, element 60; column 2, lines 1-54; and column 4, lines 1-15; 
column 8, lines 1-55). 

Regarding claim 9, Kakarala as modified further teaches the precision verification 
means (see Kakarala Figs. 9, the verification means 920 of Fig. 9 checks if there are any 
defective pixels i.e. if the first interpolation is sufficient or insufficient; column 11, lines 1-67) 
comprises edge calculation means for determining edge information for each predetermined 
region on the basis of the image signals and the color signals interpolated by the first 
interpolation means, and edge verification means for verifying the interpolation precision on the 
basis of the edge information (see column 1, lines 1-57; column 2, lines 1-35). 

7. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kakarala (US 
7,015,961 B2) in view of Acharya et al. (US 7,053,944 Bl). 

Regarding claim 8, Kakarala teaches the precision verification means comprises 
determining information for each pixel on the basis of the image signals and the color signals 
interpolated by the first interpolation means, and hue verification means for verifying the 
interpolation precision on the basis of the information. 

Kakarala does not teach hue calculation means for determining hue information for each 

pixel. 



Application/Control Number: 10/543,079 Page 10 

Art Unit: 2622 

However, Acharya et al. teaches hue calculation means for determining hue information 
for each pixel (see column 2, lines 20-50; column 3, lines 3-15). 

Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to modify Kakarala with the teachings of Acharya et al. to use the hue 
information and verify the precision based on the information. The motivation to do so would 
have been to produce high- quality color images from a subsampled color image that is better 
than simple color interpolation techniques as taught by Acharya et al. (see column 2, lines 1-7). 

8. Claims 2, 10, 13 and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

Okisu (6, 091, 862) in view of Kakarala (US 7,015,961 B2). 

Regarding claim 2, Okisu teaches an image pickup system for processing an image 
signal at each pixel which is composed of more than one color signals and one or more of the 
color signals are dropped out according to the location of the pixel, comprising: separation 
means for separating the image signals into first image signals and second image signals on the 
basis of predetermined characteristics relating to the image signals (see Fig. 3, element 15; 
column 4, lines 10-67) first interpolation means for interpolating the dropped-out color signals 
from the first image signals by a first interpolation method (see Fig. 3, element 16; column 4, 
lines 10-67); second interpolation means for interpolating the dropped-out color signals from the 
second image signals by a second interpolation method that is different from the first 
interpolation means (see Fig. 3, element 17 i.e. the border interpolation is second interpolation 
method that is different from the first interpolation method ; column 4, lines 10-67); 
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Okisu does not teach precision verification means for verifying the interpolation 
precision on the basis of the first image signals and the color signals interpolated by the first 
interpolation means for the regions of the first image signals, and verifying the interpolation 
precision on the basis of the second image signals and the color signals interpolated by the 
second interpolation means for the regions of the second image signals; and adjustment means 
for causing interpolation processing of the dropped-out color signals to be performed again from 
the image signals by the second interpolation means when insufficient interpolation was 
performed by the first interpolation means, and for causing interpolation processing of the 
dropped-out color signals to be performed again from the image signals by the first interpolation 
means when insufficient interpolation was performed by the second interpolation means, in cases 
where it is judged that the interpolation precision is insufficient. 

However, Kakarala teaches precision verification means for verifying the interpolation 
precision on the basis of the first image signals and the color signals interpolated by the first 
interpolation means for the regions of the first image signals(see Figs. 9, the verification means 
920 of Fig. 9 checks if there are any defective pixels i.e. if the first interpolation is sufficient or 
insufficient; column 11, lines 1-67), and verifying the interpolation precision on the basis of the 
second image signals and the color signals interpolated by the second interpolation means for the 
regions of the second image signals (see Figs. 9, once the verification means 920 checks the 
precision of the first interpolation and there are still defective pixels, it does a second 
interpolation in step 930 and the process continues; column 11, lines 1-67); and adjustment 
means for causing interpolation processing of the dropped-out color signals to be performed 
again from the image signals by the second interpolation means when insufficient interpolation 
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was performed by the first interpolation means, and for causing interpolation processing of the 
dropped-out color signals to be performed again from the image signals by the first interpolation 
means when insufficient interpolation was performed by the second interpolation means, in cases 
where it is judged that the interpolation precision is insufficient(see Figs. 9, once the verification 
means 920 checks the precision of the first interpolation and there are still defective pixels, it 
does a second interpolation in step 930 and the process continues; column 11, lines 1-67. 
Moreover, the verification continues from the second interpolation to the first interpolation and 
vice versa until the desired precision level is achieved). 

Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to modify Okisu with the teachings of Kakarala to interpolate the 
dropped out signals using two different types of interpolation and verify the precision. The 
motivation to do so would have been to suppress the color moire generated at edge portions and 
prevent the edge portions from being jagged and minimize the edge portions from exhibiting 
extreme variations as taught by Okisu (see column 1, lines 40-52 and column 2, lines 1-7). 

Regarding claim 10, Okisu as modified further teaches the separation means comprises 
edge calculation means for determining edge information (see Fig. 3, element 15; column 4, lines 
10-67; column 6, lines 40-67) for each predetermined region from the image signals, and image 
signal separation means for separating the image signals on the basis of the edge information 
(see Fig. 3, element 15; column 4, lines 10-67). 

Regarding claim 13, Okisu as modified further teaches control means that can control 
such that the operation of the precision verification means and the operation of the adjustment 
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means are stopped (see Kakarala Figs. 9, once the verification means 920 checks the precision of 
the first interpolation and there are no defective pixels, it does not perform a second interpolation 
in step 930 if there are no defective pixels in step 920; when it finally outputs the pixels without 
any defect that have gone through the first interpolation process in step 910; the verification 
means and the second interpolations means are not activated i.e. stopped). 

Regarding claim 17, all the limitations are addressed in claim 2. Moreover, Kakarala 
discloses an image processing program for processing an image signal at each pixel which is 
composed of more than one color signals and one or more of the color signals are dropped out 
according to the location of the pixel (sec column 5, lines 10-25). 

9. Claim 11 is rejected under 35 U.S.C. 103(a) as being unpatentable over Okisu and 
Kakarala as applied to claim 2 above, and further in view of Maenaka et al. (5,552,827). 
Regarding claim 11, Kakarala and Okisu teach the precision verification means 
comprises correlation calculation means for determining correlation information relating to the 
correlations between the respective color signals for each predetermined region on the basis of 
the image signals and the color signals interpolated by the first interpolation means (see Kakarala 
Figs. 9, the verification means 920 of Fig. 9 checks if there are any defective pixels i.e. if the first 
interpolation is sufficient or insufficient; column 11, lines 1-67), 

Kakarala and Okisu do not explicitly teach correlation verification means for verifying 
the interpolation precision on the basis of the correlation information 
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However, Maenaka teaches correlation verification means for verifying the interpolation 
precision on the basis of the correlation information (see Maenaka Fig.l, element 60; column 2, 
lines 1-54; and column 4, lines 1-15; column 8, lines 1-55). 

Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to modify Kakarala and Okisu with the teachings of Maenaka to verify 
the precision using the correlation information. The motivation to do so would have been to 
prevent a false color signal from being produced as taught by Maenaka et al. (see column 2, lines 
1-7). 

10. Claims 3, 14 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Okisu (6, 091, 862) in view of Kakarala (US 7,015,961 B2). 

Regarding claim 3, Okisu teaches an image pickup system for processing an image 
signal at each pixel which is composed of more than one color signals and one or more of the 
color signals are dropped out according to the location of the pixel, comprising: separation 
means for separating the image signals into first image signals and second image signals on the 
basis of predetermined characteristics relating to the image signals (see Fig. 3, element 15; 
column 4, lines 10-67) first interpolation means for interpolating the dropped-out color signals 
from the first image signals by a first interpolation method (see Fig. 3, element 16; column 4, 
lines 10-67); second interpolation means for interpolating the dropped-out color signals from the 
second image signals by a second interpolation method that is different from the first 
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interpolation means (see Fig. 3, element 17 i.e. the border interpolation is second interpolation 
method that is different from the first interpolation method ; column 4, lines 10-67); 

Okisu does not teach precision verification means for verifying the interpolation 
precision on the basis of the first image signals and the color signals interpolated by the first 
interpolation means for the regions of the first image signals, and verifying the interpolation 
precision on the basis of the second image signals and the color signals interpolated by the 
second interpolation means for the regions of the second image signals; and adjustment means 
for causing interpolation processing of the dropped-out color signals to be performed again from 
the image signals by the second interpolation means when insufficient interpolation was 
performed by the first interpolation means, and for causing interpolation processing of the 
dropped-out color signals to be performed again from the image signals by the first interpolation 
means when insufficient interpolation was performed by the second interpolation means, in cases 
where it is judged that the interpolation precision is insufficient. 

However, Kakarala teaches precision verification means for verifying the interpolation 
precision on the basis of the first image signals and the color signals interpolated by the first 
interpolation means for the regions of the first image signals(see Figs. 9, the verification means 
920 of Fig. 9 checks if there are any defective pixels i.e. if the first interpolation is sufficient or 
insufficient; column 11, lines 1-67), and verifying the interpolation precision on the basis of the 
second image signals and the color signals interpolated by the second interpolation means for the 
regions of the second image signals (see Figs. 9, once the verification means 920 checks the 
precision of the first interpolation and there are still defective pixels, it does a second 
interpolation in step 930 and the process continues; column 11, lines 1-67); and adjustment 
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means for causing interpolation processing of the dropped-out color signals to be performed 
again from the image signals by the second interpolation means when insufficient interpolation 
was performed by the first interpolation means, and for causing interpolation processing of the 
dropped-out color signals to be performed again from the image signals by the first interpolation 
means when insufficient interpolation was performed by the second interpolation means, in cases 
where it is judged that the interpolation precision is insufficient (see Figs. 9, once the verification 
means 920 checks the precision of the first interpolation and there are still defective pixels, it 
does a second interpolation in step 930 and the process continues; column 11, lines 1-67. 
Moreover, the verification continues from the second interpolation to the first interpolation and 
vice versa until the desired precision level is achieved); and selection means for selecting color 
signals having a higher interpolation precision between the color signals interpolated by the first 
interpolation means and the color signals interpolated by the second interpolation means (see 
Figs. 9, once the verification means 920 checks the precision of the first interpolation and there 
are still defective pixels, it does a second interpolation in step 930 and the process continues; 
column 11, lines 1-67. The output 940 is selected when it finally outputs the pixels without any 
defect; it selects the one with the highest precision). 

Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to modify Okisu with the teachings of Kakarala to interpolate the 
dropped out signals using two different types of interpolation and verify the precision. The 
motivation to do so would have been to suppress the color moire generated at edge portions and 
prevent the edge portions from being jagged and minimize the edge portions from exhibiting 
extreme variations as taught by Okisu (see column 1, lines 40-52 and column 2, lines 1-7). 
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Regarding claim 14, Okisu as modified further teaches control means that can control 
such that the operation of the second interpolation means and the operation of the precision 
verification means are stopped, and that can control such that when these operations are stopped, 
the selection means is caused to select only the color signals that are interpolated by the first 
interpolation means (see Figs. 9, once the verification means 920 checks the precision of the first 
interpolation and there are still defective pixels, it does not perform a second interpolation in 
step 930 if there are no defective pixels in step 920; column 11, lines 1-67. The output 940 is 
selected when it finally outputs the pixels without any defect that have gone through the first 
interpolation process in step 910). 

Regarding claim 18, all the limitations are addressed in claim 3. Moreover, Kakarala 
discloses an image processing program for processing an image signal at each pixel which is 
composed of more than one color signals and one or more of the color signals are dropped out 
according to the location of the pixel (sec column 5, lines 10-25). 

Conclusion 

11. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Adams, Jr. et al. (5,506,619) teaches adaptive interpolating means for missing pixels and 
selecting the one with the highest precision by comparing it with a predetermined threshold and 
if the precision value is greater than or equal to the threshold, then the default interpolation is 
selected (see column 4, lines 30-67). 
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12. Contacts 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to MEKONNEN DAGNEW whose telephone number is (571)270- 
5092. The examiner can normally be reached on Monday-Thursday, 8AM-5PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Lin Ye can be reached on (571)272-7372. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

MD 

August 14, 2008 

/Lin Ye/ 
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